Activation Function Research Articles

Insulator defect detection in transmission line based on an improved lightweight YOLOv5s algorithm

As one of the most important components on the transmission line, the insulator is prone to failure, which affects the safe operation of electrical power system. Hence, it is crucial to accurately detect the insulator defects for further maintenance in time. Recently, with the development of artificial intelligence and target detection algorithms, the insulator defect detection has received more and more attention. However, there are still existing some difficulties: insufficient samples and low detection accuracy. To improve the accuracy of insulator defect detection, this paper proposes an auto-detection method based on an improved lightweight YOLOv5s model. First, this paper introduces the basic network frame of YOLOv5s and proposes an improved algorithm by utilizing the GIoU loss function, Mish activation function, and CBAM module. Then, performs data enhancement in the insulator dataset to enhance the robustness of the model. Finally, trains and tests the improved YOLOv5s model, and compares it with traditional target detection algorithms. Compared with traditional target detection algorithms, the AP value of the proposed algorithm in detecting insulator defects can be improved by 5 %. The results demonstrate the improved algorithm proposed in this paper can effectively identify and position the insulator defects.

Hybrid synaptic structure for spiking neural network realization

Neural networks and neuromorphic computing represent fundamental paradigms as alternative approaches to Von-Neumann-based implementations, advancing in the applications of deep learning and machine vision. Nonetheless, conventional semiconductor circuits encounter challenges in achieving ultra-fast processing speed and low power consumption due to their dissipative properties. Conversely, single flux quantum circuits exhibit inherent spiking behavior, showcasing their characteristics as a promising candidate for spiking neural networks (SNNs). In this work, we present a compact hybrid synapse circuit to mimic the biological interconnect functionality, enabling the weighting operations for excitatory and inhibitory impulses. Additionally, the proposed structure facilitates input accumulation, which is performed before the activation function. In the experiments, our synaptic structure interfaces with a soma circuit fabricated using a commercial Nb process, underscoring its compatibility and supporting its potential for integration into efficient neural network architectures. The weight value on the synapse is configurable by utilizing cryo-CMOS circuits, providing adaptability to the inference networks. We’ve successfully designed, fabricated, and partially tested the JJ-Synapse within our cryocooler system, enabling high-speed inference implementation for SNNs.

A maize seed variety identification method based on improving deep residual convolutional network

Seed quality and safety are related to national food security, and seed variety purity is an essential indicator in seed quality detection. This study established a maize seed dataset comprising 5877 images of six different types and proposed a maize seed recognition model based on an improved ResNet50 framework. Firstly, we introduced the ResStage structure in the early stage of the original model, which facilitated the network’s learning process and enabled more efficient information propagation across the network layers. Meanwhile, in the later residual blocks of the model, we introduced both the efficient channel attention (ECA) mechanism and depthwise separable (DS) convolution, which reduced the model’s parameter cost and enabled the capturing of more precise and detailed features. Finally, a Swish-PReLU mixed activation function was introduced globally to improve the overall predictive power of the model. The results showed that our model achieved an impressive accuracy of 91.23% in corn seed classification, surpassing other related models. Compared with the original model, our model improved the accuracy by 7.07%, reduced the loss value by 0.19, and decreased the number of parameters by 40%. The research suggested that this method can efficiently classify corn seeds, holding significant value in seed variety identification.

ОДНОВИМІРНА ЗГОРТКОВА МОДЕЛЬ НЕЙРОННОЇ МЕРЕЖІ ДЛЯ ОБРОБЛЕННЯ СИГНАЛІВ З АМПЛІТУДНОЮ МОДУЛЯЦІЄЮ БАГАТЬОХ СКЛАДОВИХ

The processing of radio signals using artificial neural networks (ANNs) has great potential for research, which can be explained by the adaptability of ANNs to various transmission conditions and the ability to detect abstract patterns of changes in signal parameters. The article reviews the works of other authors devoted to different ways of using ANNs for processing radio signals. Taking into account the information in the reviewed works, the research task was formed, which consists in developing an optimized ANN model for radio signal processing. Signals with amplitude modulation of many components (AMMC) were chosen to form training samples for ANN. The choice of modulation type is justified by greater energy efficiency compared to other widely used digital modulation types, such as quadrature amplitude modulation. Mathematic basis of AMMC signal generation is described. The process of finding the coordinates of three component 8-AMMC signal constellation is explained, the formation of signals in the time plane based on the found coordinates is explained as well as their discretization and the addition of white noise. An iterative algorithm for generating initial data for ANN based on the described ratios is proposed. The general structure of one-dimensional convolutional neural network is considered. Functions of individual neurons, connections between them, the formation of layers and the convolution operation are described mathematically. On the basis of the previously given ratios, a final display of the network was formed. Specific dimensions and activation functions for layers are selected. The use of convolutional layers is justified by time invariance. Based on the reviewed mathematical models, selected activation functions and dimensions, a neural model was formed. The process of validating the effectiveness of the formed neural model is described, which is based on comparing the symbolic error probabilities of the proposed and reference models at different signal-to-noise ratios. The validation results are presented. The advantages of the obtained model over the previously proposed purely recurrent model and the AMMC reference receiver are explained.

Encoder–Decoder Based LSTM and GRU Architectures for Stocks and Cryptocurrency Prediction

This work addresses the intricate task of predicting the prices of diverse financial assets, including stocks, indices, and cryptocurrencies, each exhibiting distinct characteristics and behaviors under varied market conditions. To tackle the challenge effectively, novel encoder–decoder architectures, AE-LSTM and AE-GRU, integrating the encoder–decoder principle with LSTM and GRU, are designed. The experimentation involves multiple activation functions and hyperparameter tuning. With extensive experimentation and enhancements applied to AE-LSTM, the proposed AE-GRU architecture still demonstrates significant superiority in forecasting the annual prices of volatile financial assets from the multiple sectors mentioned above. Thus, the novel AE-GRU architecture emerges as a superior choice for price prediction across diverse sectors and fluctuating volatile market scenarios by extracting important non-linear features of financial data and retaining the long-term context from past observations.

ECHO: Energy-Efficient Computation Harnessing Online Arithmetic—An MSDF-Based Accelerator for DNN Inference

Deep neural network (DNN) inference demands substantial computing power, resulting in significant energy consumption. A large number of negative output activations in convolution layers are rendered zero due to the invocation of the ReLU activation function. This results in a substantial number of unnecessary computations that consume significant amounts of energy. This paper presents ECHO, an accelerator for DNN inference designed for computation pruning, utilizing an unconventional arithmetic paradigm known as online/most significant digit first (MSDF) arithmetic, which performs computations in a digit-serial manner. The MSDF digit-serial computation of online arithmetic enables overlapped computation of successive operations, leading to substantial performance improvements. The online arithmetic, coupled with a negative output detection scheme, facilitates early and precise recognition of negative outputs. This, in turn, allows for the timely termination of unnecessary computations, resulting in a reduction in energy consumption. The implemented design has been realized on the Xilinx Virtex-7 VU3P FPGA and subjected to a comprehensive evaluation through a rigorous comparative analysis involving widely used performance metrics. The experimental results demonstrate promising power and performance improvements compared to contemporary methods. In particular, the proposed design achieved average improvements in power consumption of up to 81%, 82.9%, and 40.6% for VGG-16, ResNet-18, and ResNet-50 workloads compared to the conventional bit-serial design, respectively. Furthermore, significant average speedups of 2.39×, 2.6×, and 2.42× were observed when comparing the proposed design to conventional bit-serial designs for the VGG-16, ResNet-18, and ResNet-50 models, respectively.

DEPLOYMENT OF ENHANCED DEEP LEARNING MODEL WITH THE BEST ESTIMATORS ON OPTIMIZERS AND ACTIVATION FUNCTIONS FOR HEALTHCARE IN WEB APPLICATION

Breast cancer and brain tumor stand as leading global causes of mortality. Brain tumor uses Magnetic Resonance Imaging (MRI) which offers superior clarity in visualizing brain structures compared to other imaging modalities, while Breast cancer uses ultrasonography (US) serves as a common tool for detecting breast cancer despite its inherent limitations in image quality. Motion artifacts frequently hinder MRI scans, necessitating skilled radiologists for accurate interpretation. Computer-aided diagnosis (CAD) systems driven by artificial intelligence, present a promising solution by consistently assisting radiologists in analyzing US images. Convolutional neural networks (CNNs) leverage various optimizers like Adam and Stochastic Gradient Descent (SGD), RMSprop, Adagrad, and Adadelta as well as activation functions including PReLu, LeakyReLu, Elu, and ReLu for their construction and training. The comparative analysis highlights the importance of optimizers and activation functions in deep learning algorithms for predicting brain tumors and breast cancer. The Adam optimizer combined with the ReLU activation function achieved an accuracy of 85% for breast cancer prediction, while RMSprop combined with ReLU activation function achieved a higher accuracy of 93% for brain tumor classification. From this research, considerable deep learning configurations are identified for both breast cancer and brain tumor prediction, facilitating more precise and efficient diagnoses. The comparative analysis provides valuable insights for those involved in medical imaging applications. Furthermore, The CNN model is deployed in web interface using flask framework to streamline the integration of these models into healthcare systems. This interface simplifies the input of medical data including image data and provides real-time predictions. KEYWORDS—Convolutional neural networks, Comparative Analysis, Activation functions, Optimizers, Ultrasound images, MRI images, Breast cancer, Brain tumor, medical imaging applications, FLASK.

Part-of-speech Tagging for Low-resource Languages: Activation Function for Deep Learning Network to Work with Minimal Training Data

Numerous natural language processing (NLP) applications exist today, especially for the most commonly spoken languages such as English, Chinese, and Spanish. Popular traditional methods such as Rule based methods, Naive Bayes classifiers, Hidden Markov models, Conditional Random field-based classifiers, and other stochastic methods have contributed to this improvement in the past. Recently, deep learning has led to exciting breakthroughs in several areas of artificial intelligence, including image processing and natural language processing. It is important to label words as parts of speech to begin developing most of the NLP applications. A deep study in this area reveals that many popular approaches used for this purpose require massive training data. Therefore, these approaches have not been helpful for languages not rich in digital resources. Applying these methods with very little training data prompts the need for innovative problem-solving. This article describes our research, which examines the strengths and weaknesses of well-known approaches, such as conditional random fields and state-of-the-art deep learning models, when applied for part-of-speech tagging using minimal training data for Assamese and English. We also examine the factors affecting them. We discuss our deep learning architecture and the proposed activation function, which shows promise with little training data. The activation function categorizes words belonging to different classes with more confidence by using the outcomes of statistical methods with SMTaylor SoftMax in our deep learning model. With minimal training, our deep learning architecture using the proposed modification of SM-Taylor SoftMax improves accuracy upto 4%, for our small dataset. This technique is a combination of SMTaylor SoftMax and statistical probability distribution of words over tags.

A simple theory for training response of deep neural networks

Abstract Deep neural networks give us a powerful method to model the training dataset’s relationship between input and output. We can regard that as a complex adaptive system consisting of many artificial neurons that work as an adaptive memory as a whole. The network’s behavior is training dynamics with a feedback loop from the evaluation of the loss function. We already know the training response can be constant or shows power law-like aging in some ideal situations. However, we still have gaps between those findings and other complex phenomena, like network fragility. To fill the gap, we introduce a very simple network and analyze it. We show the training response consists of some different factors based on training stages, activation functions, or training methods. In addition, we show feature space reduction as an effect of stochastic training dynamics, which can result in network fragility. Finally, we discuss some complex phenomena of deep networks.

Octave-spanning supercontinuum generation in a CMOS-compatible thin Si3N4 waveguide coated with highly nonlinear TeO2.

Supercontinuum generation (SCG) is an important nonlinear optical process enabling broadband light sources for many applications, for which silicon nitride (Si3N4) has emerged as a leading on-chip platform. To achieve suitable group velocity dispersion and high confinement for broadband SCG the Si3N4 waveguide layer used is typically thick (>∼700 nm), which can lead to high stress and cracks unless specialized processing steps are used. Here, we report on efficient octave-spanning SCG in a thinner moderate-confinement 400-nm Si3N4 platform using a highly nonlinear tellurium oxide (TeO2) coating. An octave supercontinuum spanning from 0.89 to 2.11 µm is achieved at a low peak power of 258 W using a 100-fs laser centered at 1565 nm. Our numerical simulations agree well with the experimental results giving a nonlinear parameter of 2.5 ± 0.5 W-1m-1, an increase by a factor of 2.5, when coating the Si3N4 waveguide with a TeO2 film. This work demonstrates highly efficient SCG via effective dispersion engineering and an enhanced nonlinearity in CMOS-compatible hybrid TeO2-Si3N4 waveguides and a promising route to monolithically integrated nonlinear, linear, and active functionalities on a single silicon photonic chip.

Efficient Hand Bone Segmentation for Medical Applications Using Refined DeepLab Model

In the medical field, analyzing various bone structures is crucial due to the rigid nature of bones. X-ray imaging plays an essential role in medical procedures, including bone age evaluation, fracture detection, and implant creation. However, operator involvement can introduce biases and increase processing time. Automating the process could reduce processing time and enhance diagnostic accuracy by minimizing biases and operator involvement. This paper introduces the Refined DeepLab model, a lightweight encoder–decoder-based approach for multiclass segmentation of hand bones. The primary objective is to assist physicians in tasks such as bone age analysis, fracture detection, hand movement analysis, and implant design. The research objectives are organized into three phases, with this work focusing on the first phase of our objectives, which is delineating bones from tissues, studying the bone structure, and multiclass segmentation of hand bones. The model utilizes DenseNet121 as its feature extractor and Sigmoid-weighted Linear Unit (SiLU) as its activation function. Experimental findings demonstrate promising performance in hand bone multiclass segmentation, with a Mean Intersection over Union (mIoU) of 85.02% and a Dice score of 92.2%. The comprehensive analysis of the results confirms that the proposed model excels, particularly in achieving the right balance between computational efficiency and precise segmentation.

Relationship of physical activity and cognitive functioning among breast cancer survivors: a cross-sectional analysis

IntroductionCancer related cognitive decline is a common long-term side effect of cancer and its treatments among breast cancer survivors. Physical activity is a modifiable risk factor related to cognitive decline. However, existing research lacks consensus regarding the relationship between cognition and exercise as well as the impact of cancer treatments on this relationship. Baseline data from an ongoing randomized clinical trial was utilized to examine the relationship between self-reported and objectively measured cognition with physical activity. Exploratory analyses examined cancer treatments as potential moderators.MethodsBreast cancer survivors (N = 253) completed a battery of neurocognitive tests, the PROMIS Cognitive abilities questionnaire, medical charts abstracted for treatment information, and wore an ActiGraph accelerometer at the waist for 7 days. Data were analyzed using multiple linear regression models.ResultsParticipants were on average 58.5 (SD = 8.88) years old, diagnosed 3 years prior to enrollment (SD = 1.27) with 57% treated with chemotherapy and 80% receiving hormone therapy at baseline. Better self-reported cognitive ability was significantly associated with greater min of moderate to vigorous physical activity (MVPA; β = 0.070, se = 0.028, p = 0.012). There were no significant associations with any objectively measured cognitive domains. Time since diagnosis (years) was a significant moderator of MVPA and Processing Speed (β = −0.103, se = 0.043, p = 0.017). Treatment with chemotherapy and/or hormones did not significantly moderate the relationship between MVPA and any of the cognitive measures or domains.ConclusionFindings suggest that physical activity is related to self-reported cognition but not objectively measured cognition. Greater physical activity was associated with faster processing speed in participants closer in time to their cancer diagnosis. These results emphasize the need for more research to understand when cancer survivors may benefit from physical activity and what aspects of cognition may be improved.

A Neural Network Approach for Software Reliability Prediction

The increasing reliance on computer software has raised significant concerns regarding software reliability evaluation. Over the past four decades, various software reliability models have been developed, encompassing both parametric and nonparametric approaches. However, no single model has demonstrated effectiveness in handling all types of datasets. In response to this challenge, the deep neural network (DNN), a powerful deep learning model, has emerged as a promising solution. By leveraging the flexibility and adaptability of artificial neural networks (ANN), the DNN model exhibits remarkable prediction performance by capturing training variables and exploring deeper layers. This study presents an approach that utilizes a DNN model based on an ANN architecture with multiple activation functions. This approach aims to estimate software reliability and predict the frequency of software flaws. Through extensive experimental analysis and validation, the proposed DNN model surpasses the predictive accuracy achieved by existing parametric and nonparametric models. These results highlight the potential of the DNN model in effectively addressing the complexities involved in software reliability evaluation. By combining the power of deep learning with the adaptability of ANN, the proposed model offers a promising and accurate solution for software reliability prediction.

An efficient deep learning model for tomato disease detection

Tomatoes possess significant nutritional and economic value. However, frequent diseases can detrimentally impact their quality and yield. Images of tomato diseases captured amidst intricate backgrounds are susceptible to environmental disturbances, presenting challenges in achieving precise detection and identification outcomes. This study focuses on tomato disease images within intricate settings, particularly emphasizing four prevalent diseases (late blight, gray leaf spot, brown rot, and leaf mold), alongside healthy tomatoes. It addresses challenges such as excessive interference, imprecise lesion localization for small targets, and heightened false-positive and false-negative rates in real-world tomato cultivation settings. To address these challenges, we introduce a novel method for tomato disease detection named TomatoDet. Initially, we devise a feature extraction module integrating Swin-DDETR’s self-attention mechanism to craft a backbone feature extraction network, enhancing the model’s capacity to capture details regarding small target diseases through self-attention. Subsequently, we incorporate the dynamic activation function Meta-ACON within the backbone network to further amplify the network’s ability to depict disease-related features. Finally, we propose an enhanced bidirectional weighted feature pyramid network (IBiFPN) for merging multi-scale features and feeding the feature maps extracted by the backbone network into the multi-scale feature fusion module. This enhancement elevates detection accuracy and effectively mitigates false positives and false negatives arising from overlapping and occluded disease targets within intricate backgrounds. Our approach demonstrates remarkable efficacy, achieving a mean Average Precision (mAP) of 92.3% on a curated dataset, marking an 8.7% point improvement over the baseline method. Additionally, it attains a detection speed of 46.6 frames per second (FPS), adeptly meeting the demands of agricultural scenarios.

Nondestructive testing of runny salted egg yolk based on improved ConvNeXt-T.

Salted egg yolks from salted duck eggs are widely utilized in the domestic and international food industry as both raw materials and ingredients. When salted egg yolks are not fully cured and matured, they exist in a fluid state, with a mixture of solid and liquid internally. Due to this composition, they are susceptible to deterioration during storage and usage, necessitating their detection and classification. In this study, a dataset specifically for salted egg yolks was established, and the ConvNeXt-T model, employed as the benchmark model, underwent two notable improvements. First, a lightweight location-aware circular convolution (ParC) was introduced, utilizing a ParC-block to replace a portion of the original ConvNeXt-T block. This enhancement aimed to overcome the limitations of convolution in extracting global feature information while integrating the global sensing capability of vision transformer and the localization capability of convolution. Additionally, the activation function was modified through substitution. These improvements resulted in the final model. Experimental results indicate that the enhanced model exhibits faster convergence on the custom salted egg yolk dataset compared to the baseline model. Furthermore, a significant reduction of model parameters by a factor of 4 led to a 2.167 percentage point improvement in the accuracy of the test set. The ParC-ConvNeXt-SMU-T model achieved an accuracy of 96.833% with 26.8 million parameters. Notably, the improved model demonstrates exceptional effectiveness in recognizing salted egg yolks. PRACTICAL APPLICATION: This study can be widely applied in the process of salted egg yolk production and quality inspection, which can improve the actual sorting efficiency of salted egg yolks and reduce the labor cost at the same time. It can also be used for nondestructive testing of salted egg yolks by governmental enterprises and other regulatory authorities.

Constraint‐aware optimization model for plane truss structures via single‐agent gradient descent

AbstractThis study introduces the constraint‐aware optimization model (CAOM), a novel optimization framework designed to optimize the size, shape, and topology of plane truss structures simultaneously. Unlike traditional optimization models, which rely on gradient descent and frequently struggle with managing various constraints due to their dependence on a single optimization agent, CAOM effectively addresses this challenge. It does so by incorporating a variety of assistant modules along with the Adam optimizer, a variant of the gradient descent method. Uniquely, CAOM employs the leaky rectified linear unit (ReLU) activation function beyond its conventional use in neural networks, applying it as a mechanism to integrate constraints and losses seamlessly. The model's effectiveness was validated through two numerical examples and a practical application, demonstrating that CAOM can reduce structural weight by up to 84% compared to unoptimized designs while fully adhering to structural, dimensional, and moveable constraints. Furthermore, the study found that while shape optimization plays a key role for stiffness‐governed structures, size optimization is crucial for strength‐governed structures. Optimizing size, shape, and topology together consistently leads to the most weight‐efficient designs. This emphasizes the significance of a holistic approach in the optimization processes.

Exploring the Impact of Preoperative Sleep Fragmentation on Cognitive Function in Mice: The Role of Microglial Activation and Iron Metabolism

Background. Perioperative neurocognitive disorders (PND) are a significant concern, particularly for aged individuals. Sleep fragmentation (SF), a common condition in older adults, is considered a risk factor for PND. The present study explored the impact of SF on cognitive function and its association with microglial activation and iron metabolism. Methods. Adult and aged C57BL/6J mice were subjected to tibial fracture surgery (TFS) and varying durations of SF. Cognitive function was assessed using the Morris water maze and fear conditioning experiments. Microglial activation was evaluated by measuring CD68 protein expression and inflammatory cytokine levels. Iron metabolism and ferroptosis-related proteins were also examined. Results. SF significantly impacted spatial memory and conditioned fear responses in mice, with aged mice showing greater susceptibility. Microglial activation, indicated by changes in CD68 protein expression and inflammatory cytokine levels, was observed in mice exposed to SF. Alterations in iron metabolism, as evidenced by changes in hippocampal iron content and expression of ferroptosis-related proteins, were also observed in these mice. Conclusion. SF can lead to significant cognitive impairment, particularly in aged mice, likely mediated through microglial activation and dysregulated iron metabolism. These findings provide novel insights into the pathogenesis of PND and suggest potential targets for intervention. Significance. This study illuminates the complex interactions between SF, microglial activation, and cognitive function. It highlights the importance of sleep quality for cognitive health in older adults and points to potential therapeutic strategies for preventing PND, including targeting microglial activation and iron metabolism.

Adenosine A2A receptor antagonist SCH58261 improves the cognitive function in Alzheimer's disease model mice through activation of Nrf2 via an autophagy-dependent pathway.

Adenosine, an important endogenous neuromodulator, contributes to a broad set of several neurodegenerative diseases. The adenosine A2A receptor (A2AR) is the most involved in neuropathological effects and plays an important role in the pathogenesis of Alzheimer's disease (AD). However, the effect of A2AR antagonist and the underlying mechanism in AD model mice remains unclear. The amyloid beta (Aβ)1-42-induced mice AD models were used in this study. Several behavioral experiments were performed to evaluate the improvement of AD mice treat with A2AR receptor antagonist. For mechanism analysis, autophagy-related proteins, Kelch-like ECH-associated protein1(Keap1)- nuclear factor erythroid-derived factor 2-related factor (Nrf2) pathway activation and synaptic function were studied by using western blot, immunofluorescence (IF), immunohistochemistry (IHC), transmission electron microscope (TEM), real-time quantitative (PCR) and patch clamp. Pharmacological blockade of adenosine A2AR by SCH58261 (SCH) ameliorated cognitive deficits and decreased expression levels of several AD biomarkers, including Aβ and hyperphosphorylation of Tau. Moreover, SCH activated the Nrf2 pathway through autophagy mediated Keap1 degradation, resulting in the improvement of neurons autophagy dysfunction, synaptic plasticity, and synaptic transmission. Our data clarified that the SCH58261 (an antagonist of adenosine A2A receptor) could increase the level of autophagy, promote the ability of anti-oxidative stress by the activation of Keap1-Nrf2 pathway, and improve the synaptic function in Aβ1-42-induced Alzheimer's disease mice or cell model, which provided a potential therapeutic strategy for AD. A2AR antagonism represents a promising strategy for the anti-AD agents development through autophagy-dependent pathway.

Detection of objects in satellite and aerial imagery using channel and spatially attentive YOLO-CSL for surveillance

A novel lightweight Rotational Object Detection algorithm is proposed to overcome the shortcomings of conventional computer-vision-aided object detection methods used in Remote Sensing and Surveillance that overlook the variability in size and orientation of objects in satellite and aerial images. This advanced algorithm integrates a branch dedicated to angle prediction and employs the circular smooth label (CSL) method for angle classification. This approach is suitable for scenarios that require detection in rotational boxes. Our work is further distinguished by the introduction of a novel Channel and Spatial Attention (CSA) module, which is seamlessly integrated into the YOLOv5-CSL framework via the C3CS module. This module accentuates pertinent features through both the channel and spatial attention mechanisms. In addition, bicubic interpolation and the GELU activation function were incorporated into the YOLOv5-CSLA model. Our model achieved 57.86 mAP on the challenging DOTA v2 dataset surpassing the second-best method by 0.20 points and simultaneously consuming 11 million fewer parameters and 103 fewer GFLOPs (our model consumes 25 M Params and 54 GFLOPs), justifying its suitability for deployment on a large majority of platforms, as the compute required is a challenge in real-time deployment scenarios.

Geographic heterogeneity of activation functions in urban real-time flood forecasting: Based on seasonal trend decomposition using Loess-Temporal Convolutional Network-Gated Recurrent Unit model

Urban real-time flood forecasting is crucial for flood prevention and sustainable development, but it poses challenges due to data inputs and activation functions selection in data-driven models without sufficient focus of geographic heterogeneity. In this study, a novel Seasonal Trend Decomposition using Loess (STL)-Temporal Convolutional Network (TCN)-Gated Recurrent Unit (GRU) model is proposed to improve urban real-time flood forecasting accuracy, twenty-one different activation functions are considered for geographic heterogeneity. Experiments are conducted at six urban drainage system locations in Odense, Denmark. The results show that: (1) STL effectively prepares data for forecasting using TCN and GRU models, leading improved performance compared to single models. STL-TCN-GRU deep learning model demonstrates strong applicability in urban forecasting, achieving an overall accuracy of 0.0079, 0.0140, 0.0482 and 0.9581 in Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE) and Nash-Sutcliffe efficiency coefficient (NSE), respectively. (2) Softsign emerges as the best activation function for forecasting in lower drainage system locations, with average accuracy of 0.0094, 0.0018, 0.0049 and 0.9938 in MAE, RMSE, MAPE and NSE, respectively. Furthermore, Softsign proves to be the best activation function for forecasting in middle drainage system locations, with average accuracy of 0.0047, 0.0081, 0.0309 and 0.9547 in MAE, RMSE, MAPE and NSE, respectively. Swish is the best activation function for forecasting in upper drainage system locations, with average accuracy of 0.0052, 0.0080, 0.0627 and 0.9060 in MAE, RMSE, MAPE and NSE, respectively. This study provides valuable insights for urban real-time flood forecasting modeling with high accuracy and evidence for activation function selection in data-driven models like STL-TCN-GRU for geographic heterogeneity.